{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# cuml.accel: Zero-Code Change Acceleration with NVIDIA GPUs.\n",
    "`cuml.accel` is a new tool in cuML which allows you to accelerate [Scikit-Learn](https://scikit-learn.org/1.5/index.html) estimators on NVIDIA GPUs without having to make any changes to your scripts or notebooks. The following notebook is taken directly from the [Scikit-Learn example gallery](https://scikit-learn.org/1.5/auto_examples/cluster/plot_kmeans_digits.html#sphx-glr-auto-examples-cluster-plot-kmeans-digits-py) to demonstrate how unaltered Scikit-Learn code can be accelerated with `cuml.accel`.\n",
    "\n",
    "As always, please remember to [cite](https://scikit-learn.org/1.5/about.html#citing-scikit-learn) Scikit-Learn in any publication based on the fantastic work of the Scikit-Learn community."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The following magic is the only change required to enable GPU acceleration with cuml.accel\n",
    "%load_ext cuml.accel\n",
    "# If you wish to see results WITHOUT cuml.accel, be sure to comment out the above AND restart the notebook kernel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# A demo of K-Means clustering on the handwritten digits data\n",
    "\n",
    "In this example we compare the various initialization strategies for K-means in\n",
    "terms of runtime and quality of the results.\n",
    "\n",
    "As the ground truth is known here, we also apply different cluster quality\n",
    "metrics to judge the goodness of fit of the cluster labels to the ground truth.\n",
    "\n",
    "Cluster quality metrics evaluated (see `clustering_evaluation` for\n",
    "definitions and discussions of the metrics):\n",
    "\n",
    "=========== ========================================================\n",
    "Shorthand    full name\n",
    "=========== ========================================================\n",
    "homo         homogeneity score\n",
    "compl        completeness score\n",
    "v-meas       V measure\n",
    "ARI          adjusted Rand index\n",
    "AMI          adjusted mutual information\n",
    "silhouette   silhouette coefficient\n",
    "=========== ========================================================\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the dataset\n",
    "\n",
    "We will start by loading the `digits` dataset. This dataset contains\n",
    "handwritten digits from 0 to 9. In the context of clustering, one would like\n",
    "to group images such that the handwritten digits on the image are the same.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "from sklearn.datasets import load_digits\n",
    "\n",
    "data, labels = load_digits(return_X_y=True)\n",
    "(n_samples, n_features), n_digits = data.shape, np.unique(labels).size\n",
    "\n",
    "print(f\"# digits: {n_digits}; # samples: {n_samples}; # features {n_features}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define our evaluation benchmark\n",
    "\n",
    "We will first our evaluation benchmark. During this benchmark, we intend to\n",
    "compare different initialization methods for KMeans. Our benchmark will:\n",
    "\n",
    "* create a pipeline which will scale the data using a\n",
    "  :class:`~sklearn.preprocessing.StandardScaler`;\n",
    "* train and time the pipeline fitting;\n",
    "* measure the performance of the clustering obtained via different metrics.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "from time import time\n",
    "\n",
    "from sklearn import metrics\n",
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "\n",
    "def bench_k_means(kmeans, name, data, labels):\n",
    "    \"\"\"Benchmark to evaluate the KMeans initialization methods.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    kmeans : KMeans instance\n",
    "        A :class:`~sklearn.cluster.KMeans` instance with the initialization\n",
    "        already set.\n",
    "    name : str\n",
    "        Name given to the strategy. It will be used to show the results in a\n",
    "        table.\n",
    "    data : ndarray of shape (n_samples, n_features)\n",
    "        The data to cluster.\n",
    "    labels : ndarray of shape (n_samples,)\n",
    "        The labels used to compute the clustering metrics which requires some\n",
    "        supervision.\n",
    "    \"\"\"\n",
    "    t0 = time()\n",
    "    estimator = make_pipeline(StandardScaler(), kmeans).fit(data)\n",
    "    fit_time = time() - t0\n",
    "    results = [name, fit_time, estimator[-1].inertia_]\n",
    "\n",
    "    # Define the metrics which require only the true labels and estimator\n",
    "    # labels\n",
    "    clustering_metrics = [\n",
    "        metrics.homogeneity_score,\n",
    "        metrics.completeness_score,\n",
    "        metrics.v_measure_score,\n",
    "        metrics.adjusted_rand_score,\n",
    "        metrics.adjusted_mutual_info_score,\n",
    "    ]\n",
    "    results += [m(labels, estimator[-1].labels_) for m in clustering_metrics]\n",
    "\n",
    "    # The silhouette score requires the full dataset\n",
    "    results += [\n",
    "        metrics.silhouette_score(\n",
    "            data,\n",
    "            estimator[-1].labels_,\n",
    "            metric=\"euclidean\",\n",
    "            sample_size=300,\n",
    "        )\n",
    "    ]\n",
    "\n",
    "    # Show the results\n",
    "    formatter_result = (\n",
    "        \"{:9s}\\t{:.3f}s\\t{:.0f}\\t{:.3f}\\t{:.3f}\\t{:.3f}\\t{:.3f}\\t{:.3f}\\t{:.3f}\"\n",
    "    )\n",
    "    print(formatter_result.format(*results))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run the benchmark\n",
    "\n",
    "We will compare three approaches:\n",
    "\n",
    "* an initialization using `k-means++`. This method is stochastic and we will\n",
    "  run the initialization 4 times;\n",
    "* a random initialization. This method is stochastic as well and we will run\n",
    "  the initialization 4 times;\n",
    "* an initialization based on a :class:`~sklearn.decomposition.PCA`\n",
    "  projection. Indeed, we will use the components of the\n",
    "  :class:`~sklearn.decomposition.PCA` to initialize KMeans. This method is\n",
    "  deterministic and a single initialization suffice.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.cluster import KMeans\n",
    "from sklearn.decomposition import PCA\n",
    "\n",
    "print(82 * \"_\")\n",
    "print(\"init\\t\\ttime\\tinertia\\thomo\\tcompl\\tv-meas\\tARI\\tAMI\\tsilhouette\")\n",
    "\n",
    "kmeans = KMeans(init=\"k-means++\", n_clusters=n_digits, n_init=4, random_state=0)\n",
    "bench_k_means(kmeans=kmeans, name=\"k-means++\", data=data, labels=labels)\n",
    "\n",
    "kmeans = KMeans(init=\"random\", n_clusters=n_digits, n_init=4, random_state=0)\n",
    "bench_k_means(kmeans=kmeans, name=\"random\", data=data, labels=labels)\n",
    "\n",
    "pca = PCA(n_components=n_digits).fit(data)\n",
    "kmeans = KMeans(init=pca.components_, n_clusters=n_digits, n_init=1)\n",
    "bench_k_means(kmeans=kmeans, name=\"PCA-based\", data=data, labels=labels)\n",
    "\n",
    "print(82 * \"_\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### cuml.accel Results\n",
    "If you compare the timings in the above table to those obtained without `%load_ext cuml.accel`, you should notice significant speedups for random and PCA-based initialization but a much smaller speedup for k-means++. `cuml.accel` will GPU-accelerate as much as it can, but for any missing functionality, it will fall back to CPU. Therefore, you should see runtimes that are about as good or significantly better when using `cuml.accel` compared to regular CPU execution.\n",
    "\n",
    "You will also notice that the silhouette score for results should be about as good or better than that obtained without `cuml.accel`. Even though the output may not be an exact numerical match with and without `cuml.accel`, `cuml.accel` should provide _equivalent_ results to those obtained without `cuml.accel` in the sense that the _quality_ of results should be about as good or better."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize the results on PCA-reduced data\n",
    "\n",
    ":class:`~sklearn.decomposition.PCA` allows to project the data from the\n",
    "original 64-dimensional space into a lower dimensional space. Subsequently,\n",
    "we can use :class:`~sklearn.decomposition.PCA` to project into a\n",
    "2-dimensional space and plot the data and the clusters in this new space.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "reduced_data = PCA(n_components=2).fit_transform(data)\n",
    "kmeans = KMeans(init=\"k-means++\", n_clusters=n_digits, n_init=4)\n",
    "kmeans.fit(reduced_data)\n",
    "\n",
    "# Step size of the mesh. Decrease to increase the quality of the VQ.\n",
    "h = 0.02  # point in the mesh [x_min, x_max]x[y_min, y_max].\n",
    "\n",
    "# Plot the decision boundary. For that, we will assign a color to each\n",
    "x_min, x_max = reduced_data[:, 0].min() - 1, reduced_data[:, 0].max() + 1\n",
    "y_min, y_max = reduced_data[:, 1].min() - 1, reduced_data[:, 1].max() + 1\n",
    "xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))\n",
    "\n",
    "# Obtain labels for each point in mesh. Use last trained model.\n",
    "Z = kmeans.predict(np.c_[xx.ravel(), yy.ravel()])\n",
    "\n",
    "# Put the result into a color plot\n",
    "Z = Z.reshape(xx.shape)\n",
    "plt.figure(1)\n",
    "plt.clf()\n",
    "plt.imshow(\n",
    "    Z,\n",
    "    interpolation=\"nearest\",\n",
    "    extent=(xx.min(), xx.max(), yy.min(), yy.max()),\n",
    "    cmap=plt.cm.Paired,\n",
    "    aspect=\"auto\",\n",
    "    origin=\"lower\",\n",
    ")\n",
    "\n",
    "plt.plot(reduced_data[:, 0], reduced_data[:, 1], \"k.\", markersize=2)\n",
    "# Plot the centroids as a white X\n",
    "centroids = kmeans.cluster_centers_\n",
    "plt.scatter(\n",
    "    centroids[:, 0],\n",
    "    centroids[:, 1],\n",
    "    marker=\"x\",\n",
    "    s=169,\n",
    "    linewidths=3,\n",
    "    color=\"w\",\n",
    "    zorder=10,\n",
    ")\n",
    "plt.title(\n",
    "    \"K-means clustering on the digits dataset (PCA-reduced data)\\n\"\n",
    "    \"Centroids are marked with white cross\"\n",
    ")\n",
    "plt.xlim(x_min, x_max)\n",
    "plt.ylim(y_min, y_max)\n",
    "plt.xticks(())\n",
    "plt.yticks(())\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
